Joint analysis of expression levels and histological images identifies genes associated with tissue morphology

Abstract Histopathological images are used to characterize complex phenotypes such as tumor stage. Our goal is associate features of stained tissue with high-dimensional genomic markers. We use convolutional autoencoders and sparse canonical correlation analysis (CCA) on paired histological bulk gene expression identify subsets genes whose levels in a sample correlate morphological from the corresponding image. apply our approach, ImageCCA, two TCGA data sets, find sets associated structure extracellular matrix cell wall infrastructure, implicating uncharacterized processes. specific types, including neuronal cells immune system. ImageCCA GTEx v6 data, image that capture population variation thyroid colon tissues genetic variants (image morphology QTLs, or imQTLs), suggesting regulates traits.